Skip to content

Return length of 1 for continuation bytes - #705

Merged
Lysxia merged 1 commit into
haskell:masterfrom
Vekhir:patch-1
Aug 11, 2026
Merged

Return length of 1 for continuation bytes#705
Lysxia merged 1 commit into
haskell:masterfrom
Vekhir:patch-1

Conversation

@Vekhir

@Vekhir Vekhir commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

A continuation byte is always at least one byte long; returning a length of two makes accidental out of bound reads more likely. This reverts a change in behaviour in 2.1.3+ due to 1373ce3

Fixes #704

@Bodigrim

Bodigrim commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Although I don't think there is a solid technical reason for either behaviour, I don't mind merging this, because the change of behaviour in #635 was not intentional. But that assumes that we agree that the changes negate each other and do not require a major version bump in total.

@Vekhir

Vekhir commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

This isn't a breaking change in my opinion. The behaviour for continuation bytes isn't part of the interface, it's an implementation detail that may be subject to change at any time. When using the function correctly, no change can be observed.
If anything, this is a bug fix. I think it's prudent to avoid out-of-bound reads even when the function is used incorrectly.

that assumes that we agree that the changes negate each other and do not require a major version bump in total

I do agree.

Comment thread src/Data/Text/Internal/Encoding/Utf8.hs
Comment thread src/Data/Text/Internal/Encoding/Utf8.hs Outdated
A continuation byte is always at least one byte long; returning a length of two makes accidental out of bound reads more likely.

Fixes haskell#704
@Lysxia
Lysxia merged commit c363f8b into haskell:master Aug 11, 2026
24 checks passed
@Vekhir
Vekhir deleted the patch-1 branch August 11, 2026 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unsafe iter can lead to out of bounds memory access on invalid values

3 participants